Moving picture playback apparatus

ABSTRACT

Based on the event detected by event detection unit  3 , scene definition unit  4  defines the segment from the frame which is one screen of the moving picture corresponding to the event to the end of the next event as the scene The scene information of the defined scene is stored in scene information storage unit  5 , and moving picture display unit  9  reads and displays the moving picture stored in moving picture storage unit  2  based on the scene information stored in scene information storage unit  5.

This application is a divisional of application Ser. No. 09/307,026,filed on May 7, 1999 now U.S. Pat. No. 6,571,052, which is a divisionalof application Ser. No. 09/242,493 filed on Feb. 19, 1999 now U.S. Pat.No. 6,625,383, for which priority is claimed under 35 U.S.C. §120.Application Ser. No. 09/242,493 is the national phase of PCTInternational Application No. PCT/JP98/03064 filed on Jul. 8, 1998 under35 U.S.C. §371. The entire contents of each of the above-identifiedapplications are hereby incorporated by reference.

TECHNICAL FIELD

This invention relates to a moving picture edit processing for providingmoving pictures by shooting, recording and editing them with a videocamera, etc., and more particularly, this invention relates to a movingpicture playback apparatus for playing back and displaying movingpictures, and a moving picture collection apparatus for collectingmoving pictures.

BACKGROUND OF THE INVENTION

As a general and conventional moving picture playback apparatus forplaying back moving pictures, there is a video tape recorder (VTR). WithVTR, moving pictures are recorded on a video tape. When a user pressesbuttons such as “playback”, “stop”, “frame forward”, “frame rewind”etc., on the VTR apparatus, which the video tape is set to, the tape iscontrolled according to an instruction, and the recorded moving picturesare displayed on a screen.

Furthermore, as the general and conventional moving picture collectionapparatus for collecting moving pictures, there is a video camerarecorder. The video camera recorder is a video camera combined with theVTR in one piece so as to make it easy to carry, and moving picturesshot with the video camera are to be recorded on the video tape by theVTR.

Here, let us consider a case when we shoot moving pictures with thevideo camera, and record and edit the moving pictures and produce avideo title.

On the video tape, several scenes, for example, a “Scene at theAirport”,a “Scene at the Port”,a “Scene on a Superhighway”,a “Scene onthe Railroad”,etc., are recorded. Suppose if we are to produce a videotitle regarding ground transportation, we have to select appropriatescenes such as the scenes at the airport, the scenes on thesuperhighway, the scenes on the railroad, etc., from the video tape, andthen rearrange and concatenate them in a desired order. Moreover, inorder to playback a certain segment of a scene, we have to set aplayback segment in the scene, and we have to implement processing so asto specify the playback speed in order to playback the part in slowmotion. For additional effects, such as overlapping the scene of therailroad on the scene of the superhighway, several scenes must becomposited.

As an example of moving picture edit apparatus designed for such editingbased on the moving pictures recorded on the video tape, there is avideo edit system which combines the edit control apparatus with two ormore VTRs. This system designates one VTR as an input VTR and the otherVTRs as output VTRs, and edit operation such as processing orconcatenating scenes are carried out by dubbing necessary scenes fromthe input VTR to output VTR in the order. Also, a composite editoperation is implemented by compositing signals from several input VTRsby the edit control unit and then sending out to the output VTRs.

Those are examples of the conventional and general moving pictureplayback apparatus and the moving picture edit apparatus using it, inrecent years, a moving picture edit apparatus using a computer called anon-linear edit system is widely put to use (a typical product of thenon-linear edit system is the MCXpress by Avid Technology Company of theUnited States). The non-linear edit system applies the technology whichdigitallizes moving picture data, compresses the data, and stores it ina computer memory, and then reads the data as necessary and playbacksafter expanding the data. When this non-linear edit system is comparedwith the previously mentioned video edit system, the non-linear editsystem has an advantage in editing without repeating dubbing from thevideo tape to the video tape, and thereby it's usage is graduallyspreading.

The operation of the non-linear edit system, which is one example of theconventional moving picture edit apparatus, will be explained next.

FIG. 38 shows a general configuration of the conventional non-linearedit system.

The non-linear edit system is configured with a sequential access typemoving picture storage unit 2 (the moving picture storage unit for VTR,etc., and called as the sequential access type moving picture storageunit hereinafter in order to distinguish it from the moving picturestorage unit using a hard disk of a computer.) connected to a computer50. In a typical configuration of the non-linear edit system, computer50 and sequential access type moving picture storage unit 2 areconnected with a signal cable and control cable. The signal cable inputssignals of moving pictures or sound played back by sequential accesstype moving picture storage unit 2 into computer 50. And the controlcable is used for controlling playback, stop, frame forward, or framerewind from computer 50 to sequential access type moving picture storageunit 2, and for transmitting the frame number of the frame currentlydisplayed from sequential access type moving picture storage unit 2 tocomputer 50.

Moving pictures shot with the video camera are recorded on a medium suchas the video tape, and then stored in the sequential access type movingpicture storage unit 2.

The user performs editing by inputting a command from a command inputunit 8 using a keyboard or a mouse of the computer.

An example of user interface for editing is illustrated in FIG. 39.

In the figure, there are shown a scene menu window 111, a displayedscene icon 112, a moving picture playback window 113 for playing backone of the scenes, playback control buttons 114 for controlling movingpicture playback by the moving picture playback window 113, a stopbutton 115, a playback button 116, a frame rewind button 117, a frameforward button 118, a scene definition window 119 for specifying astarting frame and an ending frame for each scene, a new scene displaybutton 120 for displaying new scenes, and a moving picture edit window121 for concatenating and compositing each scene and defining a newscene.

The editing is performed in the following procedure using the userinterface aforementioned.

(1) Defining a Scene

The user firstly defines a scene of related segment in the movingpictures such as the scene of the airport, the scene of thesuperhighway, and so forth, by a scene definition unit 20. Theinformation of the defined scene is stored in scene information storageunit 5. The scene information contains a specified starting frame numberand ending frame number for each scene.

FIG. 40 indicates one example of the scene information.

The explanation of the frame number is now given. A moving picture, forinstance, consists of 30 images per second. Each image is referred to asthe frame. Moving pictures are recorded on a medium such as the videotape by sequential access type moving picture storage unit 2, and thenumber specifying each frame on the medium is called the frame number.

Business VTRs use the number called the time code for the purpose ofspecifying frames. The time code is expressed with an 8-digit numbersuch as 01:10:35:08, which represents the 8th frame of one hour, tenminute, and 35 seconds.

Defining a scene is, for example, carried out as below.

First of all, in FIG. 39, a new scene display button 120 is clicked withthe mouse. Then, a new scene having no defined title, starting frame andending frame is displayed on moving picture playback window 113. Thescene is then defined by inputting the title, starting frame number, andending frame number at the keyboard on scene definition window 119.

For defining a scene in the previously mentioned method, the user has toknow the starting frame number and the ending frame number of the sceneto be defined beforehand. Business VTRs are provided with a function todisplay the time code of the frame currently displayed on the monitorscreen, which enables the user to know the starting frame number and theending frame number by reading the displayed time code from the screenby positioning the desired frame using such buttons as the forwardbutton, rewind button, frame forward, frame rewind, etc., of the VTR.

(2) Reading a Moving Picture

Then, the defined scene is digitally compressed by a moving picturedigital compression unit 6, and stored in direct access type movingpicture storage unit 7 of computer 50 (a hard disk memory, and isreferred to as the direct access type moving picture storage unithereinafter to distinguish it from sequential access type moving picturestorage unit 2 of the VTR, etc.).

With regard to the format of digital compression of moving pictures,there are various types of formats such as MPEG, Motion-JPEG, Cinepak,and so forth, but because the computation amount becomes huge whicheversystem is used for digital compression, it is common to execute byinserting an extension board special for digital compression into thecomputer so as to shorten the processing time.

The processing for digital compression of moving pictures becomes asexplained below.

The system first performs processing of digital compression for eachscene defined by scene definition unit 20. Because each scene has itsstarting frame and ending frame described, computer 50 controlssequential access type moving picture storage unit 2 to position thestarting frame of a target scene.

Then, the playback of sequential access type moving picture storage unit2 is started, and at the same time, an instruction is sent to movingpicture digital compression unit 6, and the processing for digitalcompression starts. Moving picture digital compression unit 6 inputssignals output from sequential access type moving picture storage unit2, and converts each frame into digital data, and performs datacompression. The obtained digital compression data is stored in directaccess type moving picture storage unit 7.

(3) Displaying a Scene

In FIG. 39, scene menu window 111, for each scene stored in a sceneinformation storage unit 5, moving picture display unit 9 creates ascene icon 112 and displays. For scene icon 112, any image which allowsthe user to easily grasp the content of the scene can be used, but it istypically created by contracting the first frame of each scene.Furthermore, when one of the scenes is selected by the click of themouse, moving picture display unit 9 displays the selected scene on amoving picture playback window 113. The display of moving pictureplayback window 113 can be controlled by playback control button 114,and the playback of the moving pictures is started when playback button116 is clicked, and the playback of the moving pictures is stopped whenstop button 115 is clicked. Or, when frame rewind button 117 or frameforward button 118 is clicked, a frame immediately before or after thecurrent frame is displayed and stopped.

In such a case, moving picture display unit 9 reads the moving picturedata digitally compressed and stored in direct access type movingpicture storage unit 7, expands the data on the spot, and performsprocessing of displaying the obtained data on the screen.

The edit operation explained below is proceeded by displaying andchecking the contents of each scene as thus far explained.

(4) Executing an Edit Operation

The edit operation involves processing, concatenating, and compositingscenes. As an example of a scene processing, there are a setting of aplayback area or partial change in playback speed. This operation isperformed by specifying the playback area by the frame number on scenedefinition window 119. Also, a scene concatenation or scene compositionis performed by lining each scene icon 112 on moving picture edit window121. On moving picture edit window 121, the horizontal axis correspondsto the time axis, and when scene icon 112 is lined within this window inorder, each scene is concatenated from left to right in order and thenew scene is defined. Moreover, there are two tracks A and B on movingpicture edit window 121, and when scenes are lined at the same time belton both tracks, two scenes are composited.

The defined scene is stored in direct access type moving picture storageunit 7, and the scene information is stored in scene information storageunit 5. An example of describing the newly defined scene is shown inFIG. 41.

(5) Creating a Moving Picture

The new scene is defined by the above mentioned edit operation. Based onthe description of the scene, moving picture creation unit 11 reads themoving picture data of each scene stored in direct access type movingpicture storage unit 7, and processes, concatenates, composites, andperforms other operations to create a new moving picture data. Thecreated moving picture data is again stored in direct access type movingpicture storage unit 7. When the moving picture data is read and copiedonto a medium such as a CD ROM, the medium can be distributed and datacan be played back on other computers. It is also possible that themoving picture data is played back and the signal can be transmitted tosequential access type moving picture storage unit 2, and then recordedonto the medium such as the video tape, then the medium can bedistributed and the data can be played back on the VTR, and so on.

The conventional moving picture playback apparatus and the movingpicture collection apparatus are configured as above mentioned. Let ussuppose now that one hundred scenes are shot with the video camera, andeach scene is about 30 seconds, and then edit those moving pictures. Inorder to perform editing, scenes in the moving pictures must first bedefined. In order to define the scenes, a work of specifying, thestarting frame and ending frame for each of the 100 scenes must beperformed. For defining one scene, if it takes about 30 seconds toperform the operation for finding out the starting fame and ending frameby implementing the forward, rewind, frame forward, frame rewind, etc.of the VTR, then, it amounts to about 50 minutes to define 100 scenes.

Also, for the processing of compressing data by digitallizing the movingpictures, because it generally takes about the same time period as thecontinuation time period of the moving pictures, it takes about 50minutes for processing digitally compressing 100 scenes and storing inthe direct access type moving picture storage unit.

In this way, the conventional moving picture playback apparatus and themoving picture collection apparatus pose a problem of taking substantialtime period until the edit operation is enabled after the shooting ofmoving pictures has ended, in order to define the scenes or to read themoving pictures.

Or, in defining the scene, the work of specifying the starting frame andending frame of the scene exactly is a troublesome task which cannot bedone without careful attention.

Furthermore, the moving picture data must be digitally compressed andstored in the computer memory, if, for instance, data compression isperformed in the MPEG1 compression format, the digitally compressedmoving picture data becomes about 1.5 megabit data amount per second. Inorder to digitallize all 100 scenes and store in the computer memory inthe MPEG1 format, a storage capacity of at least 500 mega bytes isrequired.

Also, in order to perform digital compression of moving pictures, anexpansion board special for digital compression of moving pictures mustbe inserted into the computer.

In this way, because the conventional moving picture edit apparatusneeds a large capacity storage or a compression board special for movingpictures, it poses problems that it becomes a large-size apparatus notsuited to carry around or for use in a limited space such as a home oroffice

This invention is designed to solve above-mentioned problems, and itaims at obtaining a moving picture playback apparatus or a movingpicture collection apparatus which can define scenes or read movingpictures in a short time, and confirm the contents of the collectedmoving pictures quickly.

SUMMARY OF THE INVENTION

The first aspect of the invention is a moving picture playback apparatuswhich comprises

a moving picture storage unit for storing a moving picture shot by ashooting device;

an event detection unit for detecting an event of one of a change in ashooting subject, the change in the shooting device, and an operation ofa shooter during shooting, while shooting with the shooting device;

a scene definition unit for defining a segment, as a scene, separated bya frame which is a screen of the moving picture corresponding to anevent and by a frame corresponding to a next event based on the eventdetected by the event detection unit;

a scene information storage unit for storing scene information of thescene defined by the scene definition unit;

a command input unit for inputting an instruction from a user; and

a moving picture display unit for reading and displaying the movingpicture stored in the moving picture storage unit based on theinstruction input by the command input unit and the scene informationstored in the scene information storage unit.

The second aspect of the invention is the moving picture playbackapparatus, wherein the event detection unit may detect, as the event,one of the change in a position of the shooting device, direction of theshooting device, and a lens condition during shooting.

The third aspect of the invention is the moving picture playbackapparatus, wherein the event detection unit may detect, as the event,one of an operation of a scene start and an operation of a scene end ofthe shooter during shooting.

The fourth aspect of the invention is the moving picture playbackapparatus which may further comprise:

an edit unit for reading the moving picture stored in the moving picturestorage unit based on the instruction input by the command input unitand the scene information stored in the scene information storage unit,and processing, compositing, and concatenating the moving pictures toedit a new scene, and storing scene information of the new scene in thescene information storage unit; and

a moving picture creation unit for creating a moving picture for the newscene based on the scene information of the new scene stored in thescene information storage unit.

The fifth aspect of the invention is a moving picture playback apparatuswhich may comprise:

a moving picture storage unit for storing a plurality of moving picture;

a scene definition unit for defining a segment in the moving picturestored in the moving picture storage unit as a scene;

a scene information storage unit for storing scene information of thescene defined by the scene definition unit;

a typical frame storage unit for storing a typical frame whichrepresents the scene defined by the scene definition unit;

a typical frame determination unit for selecting the typical frame,which represents the scene, in the scene defined by the scene definitionunit, reading the selected typical frame from the moving picture storageunit, and storing in the typical frame storage unit;

a command input unit for inputting an instruction from a user; and

a moving picture display unit for displaying the typical frame stored inthe typical frame storage unit based on the instruction input by thecommand input unit.

The sixth aspect of the invention is the moving picture playbackapparatus, wherein the moving picture display unit may display thetypical frame from the typical frame storage unit if the frame to bedisplayed is the typical frame, and display the closest typical framebefore the typical frame to be displayed from the typical frame storageunit if the frame to be displayed is not the typical frame.

The seventh aspect of the invention is the moving picture playbackapparatus, wherein the moving picture display unit may display the frameto be displayed from the moving picture storage unit if the frame to bedisplayed is not the typical frame when the instruction input from thecommand input unit may be one of a stop, frame forward, and framerewind.

The eighth aspect of the invention is the moving picture playbackapparatus, wherein the moving picture display unit may display thetypical frame from the typical frame storage unit if the frame to bedisplayed may be the typical frame when the instruction input from thecommand input unit may be one of the stop, frame forward, and framerewind.

The ninth aspect of the invention is the moving picture playbackapparatus may further comprise an event detection unit for detecting anevent of one of a change in a shooting subject, the change in a shootingdevice, and an operation by a shooter during shooting with the shootingdevice, and

wherein the typical frame determination unit may select a framecorresponding to the event detected by the event detection unit as thetypical frame of the scene including the frame.

The tenth aspect of the invention is the moving picture playbackapparatus may further comprise an edit unit for reading the movingpicture stored in the moving picture storage unit, process, composite,and concatenate the moving pictures based on the instruction input bythe command input unit and the typical frame stored in the typical framestorage unit, and edit a new scene, and store scene information of thenew scene in the scene information storage unit, and a moving picturecreation unit for creating a moving picture for the new scene based onthe scene information of the new scene stored in the scene informationstorage unit.

The eleventh aspect of the invention is a moving picture collectionapparatus may comprise:

an event detection unit for detecting an event of one of a change in ashooting subject, the change in a shooting device, and an operation of auser during shooting with a shooting device; and

a frame number obtainment unit for obtaining a frame number of a framewhich may be one screen of the moving picture corresponding to theevent; and

an event information storage unit for storing event information whichmay correspond to the event detected by the event detection unit and theframe number.

The twelfth aspect of the invention is the moving picture collectionapparatus, wherein the event detection unit may detect, as the event,one of the change in a position of the shooting device, the change in adirection of the shooting device, the change in a lens condition, anoperation of a recording start of a user and an operation of a recordingend of the user.

The thirteenth aspect of the invention is the moving picture collectionapparatus, wherein the event information storage unit may obtain animage data of a frame in the moving picture corresponding to the event,and may obtain the change in the position of the shooting devicecorresponding to the event, direction of the shooting device and thechange of the lens condition corresponding to the event, and store asthe event information.

The fourteenth aspect of the invention is the moving picture collectionapparatus, wherein the event information unit may input and store anexplanation sentence added by the user corresponding to the event storedin the event information storage unit.

The fifteenth aspect of the invention is the moving picture collectionapparatus may further comprise an explanation sentence input unit forobtaining the image data of the frame in the moving picturecorresponding to the event stored in the event information storage unit,displaying the image data of the frame on a display screen, and havingthe user input an explanation sentence of the event corresponding to theimage data of the frame displayed on the display screen, and

wherein the event information storage unit may store the explanationsentence from the explanation sentence input unit.

The sixteenth aspect of the invention is the moving picture collectionapparatus, wherein the event detection unit may detect, as the event, anapproach of the shooting device toward a preset shooting subject.

The seventeenth aspect of the invention is the moving picture collectionapparatus, wherein the event detection unit may detect, as the event,one of an entering, and a coming out of the shooting device to/of apreset shooting area.

The eighteenth aspect of the invention is the moving picture collectionapparatus may further comprise a shooting device control unit forsending an order to the shooting device to change its direction towardthe shooting subject corresponding to the event of approaching of theshooting device toward the preset shooting subject corresponding to theevent detected by the event detection unit.

The nineteenth aspect of the invention is the moving picture collectionapparatus, wherein the shooting device control unit may send an order ofone of a recording start and a recording end corresponding to the eventof one of entering and coming out of the shooting device to/of thepreset shooting area.

The twentieth aspect of the invention is the moving picture collectionapparatus may further comprise an event information display unit fordisplaying the event information, a command input unit for inputting aninstruction from a user based on the displayed event information, and amoving picture playback instruction unit for selecting the event basedon the instruction input by the command input unit, and specifying theframe number of the frame in the moving picture corresponding to theselected event, and instructing a playback of the moving picture fromthe frame specified by the frame number.

The twenty-first aspect of the invention is a moving picture collectionapparatus may comprise:

a moving picture input unit for inputting a moving picture shot by ashooting device;

an event detection unit for detecting an event one of a change in ashooting subject, the change of the shooting device, and an operation bya user during shooting with the shooting device;

an event information storage unit for expressing information of theevent detected by the event detection unit with image and sound, andcompositing the event information with the moving picture input by themoving picture input unit and storing it.

The twenty-second aspect of a moving picture playback apparatus maycomprise

a priority set unit for setting a priority for a frame of a movingpicture composed of a plurality of continuous frames;

a priority obtainment unit for obtaining the priority set for the frameof the moving picture by the priority set unit;

a playback speed set unit for setting a moving picture playback speed atwhich the moving picture is played back;

a frame selection unit for selecting a frame of the moving picture basedon the playback speed set by the playback speed set unit and thepriority set for the frame of the moving picture obtained by thepriority obtainment unit;

a frame obtainment unit for obtaining an image of the frame selected bythe frame selection unit; and

a moving picture display unit for displaying the image of the frameobtained by the frame obtainment unit.

The twenty-third aspect of the invention is a moving picture playbackapparatus may comprise:

a priority set unit for setting a priority for a frame of a movingpicture composed of a plurality of continuous frames;

a frame obtainment unit for obtaining an image of the frame of themoving picture;

a priority obtainment unit for obtaining the priority set for the frameof the moving picture by the priority set unit;

a typical image obtainment unit for obtaining the image of the frame seta high priority by the priority set unit as a typical image;

a typical image storage unit for storing the typical image obtained bythe typical image obtainment unit;

a playback speed set unit for setting a moving picture playback speed atwhich the moving picture is to be played back;

a frame selection unit for selecting the frame of the moving picturebased on the playback speed set by the playback speed set unit and thepriority set for the moving picture frame obtained by the priorityobtainment unit; and

a moving picture display unit for displaying the image of the frameselected by the frame selection unit from one of the frame obtainmentunit and the typical image storage unit.

The twenty-fourth aspect of the invention is the moving picture playbackapparatus, wherein the moving picture display unit may read and displaythe image of the frame from the typical image storage unit when theplayback speed set by the playback speed set unit may exceed a normalplayback speed, and read and display the image of the frame from theframe obtainment unit when the playback speed may be below the normalplayback speed.

The twenty-fifth aspect of the invention is the moving picture playbackapparatus, wherein the typical image storage unit may change the frameto be stored as the typical image according to a timing relation betweenthe frame of the typical image stored by the typical image storage unitand the frame currently displayed by the moving picture display unit.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a configuration of a moving picture playback apparatus ofEmbodiment 1.

FIG. 2 is a flowchart showing the operation of a scene definition unitof Embodiment 1.

FIG. 3 shows a configuration of a moving picture playback apparatus ofEmbodiment 2.

FIG. 4 is a flowchart showing the operation of a moving picture displayunit of Embodiment 2.

FIG. 5 is a flowchart showing the flow of frame display processing ofthe moving picture display unit of Embodiment 2.

FIG. 6 shows a configuration of a moving picture playback apparatus ofEmbodiment 3.

FIG. 7 shows a configuration of a moving picture playback apparatus ofEmbodiment 4.

FIG. 8 shows a configuration of a moving picture collection apparatus ofEmbodiment 5.

FIG. 9 is a diagram explaining an event information stored in an eventinformation storage unit of Embodiment 5.

FIG. 10 is a diagram explaining an index displayed on a display unit ofEmbodiment 5.

FIG. 11 is a flowchart showing the operation of a moving picturecollection apparatus of Embodiment 5.

FIG. 12 is a flowchart showing the operation of the moving picturecollection apparatus of Embodiment 5.

FIG. 13 shows a configuration of a moving picture collection apparatusof Embodiment 6.

FIG. 14 is a diagram explaining an event information stored in an eventinformation storage unit of Embodiment 6.

FIG. 15 is a diagram explaining an index displayed on a display unit ofEmbodiment 6.

FIG. 16 shows a configuration of a moving picture collection apparatusof Embodiment 7.

FIG. 17 is a diagram explaining an example of screen of a set input unitof Embodiment 7.

FIG. 18 is a diagram explaining an event information stored in an eventinformation storage unit of Embodiment 7.

FIG. 19 is a diagram explaining an index displayed on a display unit ofEmbodiment 7.

FIG. 20 is a diagram explaining position information attachment filterof Embodiment 8.

FIG. 21 is a front view of the position information attachment filter ofEmbodiment 8.

FIG. 22 shows a configuration of a moving picture playback apparatus ofEmbodiment 9.

FIG. 23 is a diagram showing an example of setting priority ofEmbodiment 9.

FIG. 24 is a diagram showing an operation flow of a priority set unit ofEmbodiment 9.

FIG. 25 is a diagram showing an example of screen display of thepriority set unit of Embodiment 9.

FIG. 26 is a diagram showing an example of a scene configuration andframe number of Embodiment 9.

FIG. 27 is a diagram showing an example of screen display of a playbackspeed setting of Embodiment 9.

FIG. 28 is a diagram showing an example of screen display of theplayback speed setting of Embodiment 9.

FIG. 29 is a diagram showing the screen display of playback speedsetting of Embodiment 9.

FIG. 30 is a diagram showing an operation flow of a moving picturedisplay of Embodiment 9.

FIG. 31 is the diagram showing the operation flow of the moving picturedisplay of Embodiment 9.

FIG. 32 is the diagram showing the operation flow of the moving picturedisplay of Embodiment 9.

FIG. 33 shows a configuration of a moving picture playback apparatus ofEmbodiment 10.

FIG. 34 is a diagram showing an example of typical moving picture dataof Embodiment 10.

FIG. 35 is a diagram showing an operation flow of Embodiment 10.

FIG. 36 is the diagram showing the operation flow of Embodiment 10.

FIG. 37 is the diagram showing the operation flow of Embodiment 10.

FIG. 38 shows a configuration of a conventional moving picture playbackapparatus.

FIG. 39 explains a user interface of the conventional moving pictureplayback apparatus.

FIG. 40 explains a scene information of the conventional moving pictureplayback apparatus.

FIG. 41 explains the scene information of the conventional movingpicture apparatus.

DETAILED DESCRIPTION OF THE INVENTION

The embodiments of this invention will be explained below. The movingpicture playback apparatus of all embodiments are provided with an editfunction for editing moving pictures anew by processing, compositing, orconcatenating the moving pictures.

Although the explanation is given with the video camera as the apparatusfor shooting moving pictures for the following embodiments, there areimage taking apparatuses such as a digital camera, digital video camera,other than the video camera as the apparatus for shooting movingpictures.

Embodiment 1

FIG. 1 shows the configuration of a moving picture playback apparatusfor Embodiment 1.

In the figure, a video camera 1 for shooting a shooting subject, asequential access type moving picture storage unit 2 for storing movingpictures shot by video camera 1, an event detection unit 3 for detectingan event during shooting, a scene definition unit 4 for defining a scenefrom a frame which is one screen of the moving pictures corresponding tothe event detected by event detection unit 3 to the frame correspondingto the next event as the end as one segment, a scene information storageunit 5 for storing the scene information consisting of numbers of thestarting and ending frames of the scene defined by scene definition unit4, a moving picture digital compression unit 6 for compressing the scenedefined by scene definition unit 4, a direct access type moving picturestorage unit 7 for storing the compressed scene, a command input unit 8for inputting user instructions, a moving picture display unit 9 fordisplaying a scene and frame to be displayed, an edit operationexecution unit 10 for editing a new scene by processing, compositing, orconcatenating scenes, a moving picture creation unit 11 for creatingmoving pictures based on the information of the new scene, and acomputer 12 are illustrated.

As stated above, the moving picture playback apparatus of thisembodiment is configured by video camera 1 attached with sequentialaccess type moving picture storage unit 2 and event detection unit 3,and connected computer 12.

The operation of this embodiment is explained below.

(0) Shooting

The user shoots shooting subjects using video camera 1. The video camera1 is provided with a unit which transmits data by communicating withcomputer 12. As the unit which transmits data between video camera 1 andcomputer 12, there is such a communication standard as IEEE1394, butnaturally it is not limited. Although video camera 1 and computer 12 arenormally connected with a cable, in order for the cable between videocamera 1 and computer 12 not to become an obstacle, a configuration ofdata transmission by radio is more convenient. In order to make it easyto carry the apparatus, a notebook type or schedule book type computer12 is desirable.

This embodiment is a case when configured by video camera 1 equippedwith event detection unit 3 and computer connected to this. However,when event detection unit 3 is equipped with the side of computer 12,computer 12 performs transmission between video camera 1, and it ispossible to obtain the frame and the frame number being shot by videocamera 1 or being played back at an arbitrary point.

Event detection unit 3 detects the change of shooting subjects, changeof video camera 1, or events in the operation of the person takingpictures, during shooting. Here, as an example of the change of shootingsubjects, an appearance of the shooting target, the deletion of theshooting target, the movement of the shooting target, the stop of theshooting target, and the deformation of the shooting target, and so onmay be considered. And as examples of the change of video camera 1,there are the movement of the position of the camera, the change of thedirection of the camera, and zoom in or zoom out of the camera, and soon. As an example of the operation of the person who shoots pictures,the starting or interrupting of shooting, and so on may be considered.

Normally, scenes shot with video camera 1 remarkably change before orafter the frame to which the event described previously occurs.Therefore, if scenes are defined by detecting the occurrence of theseevents, and by making the frame corresponding to them as an end, theuser needs not specifying the starting frame and ending frame of eachscene one by one.

In order to detect these events, for instance, the followingconfiguration is advisable.

(0-1) Detecting the Change of Shooting Subject

In order to detect the change of shooting subject, the pictures beingshot by video camera 1 are monitored, and the difference between the twoadjacent frames can be detected when it exceeds a certain amount. Forthis purpose, it is possible to configure the moving pictures stored insequential access type moving picture storage unit 2 are checked so thatan occurrence of change is detected, or with a special apparatus using amicon incorporated in video camera 1, it is possible to read a framebeing shot into the memory one by one so that an occurrence of change isdetected. This embodiment is a case which is configured by video camera1 attached with event detection unit 3 and computer 12 connected tothis. However, when event detection unit 3 is attached to the side ofcomputer 12, it is possible to configure that frames being shot are readinto the memory of computer 12 one by one and checked so that anoccurrence of change is detected.

(0-2) Detecting the Change of Video Camera

At an arbitrary point, the state of video camera 1 can be described bythe position of the camera, the direction of the camera, and the focaldistance of the lens. By monitoring these parameters during shooting,changes can be detected.

The position of the camera and the direction of the camera, forinstance, can be calculated by attaching a magnetic sensor to videocamera 1. Because the focal distance of the lens changes as operation isperformed for zoom in and zoom out, the state of the zoom in or zoom outat the point can be detected by a mechanical unit.

This embodiment is a case which is configured by video camera 1 attachedwith event detection unit 3 and computer 12 connected to this. However,when event detection unit 3 is attached with the side of computer 12,obtained each parameter previously mentioned is read into computer 12and processed, and the time when there is a significant change in theparameters can be detected.

(0-3) Detecting the Operation of the Person Taking Pictures

Buttons for starting or interrupting the shooting are usually providedwith video camera 1, and the person who shoots can start or interruptthe shooting by pressing these buttons. Other than the events which arepreviously mentioned starting or interrupting shooting, it is possibleto define the starting or ending zoom as the events. Furthermore, inorder to separate the scene obtained by one shooting operation from thestarting of the shooting to the interrupting of the shooting into aplurality of scenes, a dedicated button is provided with video camera 1,and the person who shoots can separate the scene by pressing the buttonas the person desires to separate the scene during shooting.

In order to detect these events, it is possible to configure to detectwhen the button is pressed on video camera 1 and the event of pressingthe button is transmitted to computer 12.

(1) Defining the Scene

Scene definition unit 4 performs definition of scene based on theinformation of the event detected by event detection unit 3.

FIG. 2 is a flowchart showing the processing flow of scene definitionunit 4.

The processing procedure of the scene definition unit is explained.

First, the processing starts at step S1, and at step S2, “1” issubstituted as the scene number SN that is to be defined next and thecurrent frame number is substituted as the starting frame number FS.Whether or not an event has occurred is checked at step S3, and themoving pictures are forwarded until an event is confirmed. When theevent is confirmed, the frame number corresponding to the event is readat step S4, and the frame number (current frame number) is substitutedas the ending frame number FE.

At step S5, the frame of the starting frame number FS is assumed to bethe starting frame, and the frame of the ending frame number FE isassumed to be the ending frame and stored in scene information storageunit 5. At steps 6, 1 is added to the scene number SN, which is to bedefined next, and the prior ending frame number FD is added by 1 andsubstituted as the starting frame number FS, and returns to step S3again and waits for an occurrence of a new event.

In this way, scene definition unit 4 defines each scene with the framecorresponding to the event as the end, and the scene information, e.g.,the starting frame number and the ending frame number are written intoscene information storage unit 5.

(2) Reading the Moving Pictures

This is basically the same as the conventional non-linear editing systemoperation.

(3) Displaying the Scene

This is basically the same as the conventional non-linear editing systemoperation.

(4) Executing the Edit Operation

This is basically the same as the conventional non-linear editing systemoperation.

(5) Creating the Moving Pictures

This is basically the same as the conventional non-linear editing systemoperation.

According to the present embodiment, as explained above, the user neednot do cumbersome work which requires an attentive power to instruct thestarting frame and the ending frame of each scene manually, and also cansave time.

Embodiment 2

FIG. 3 shows a configuration of a moving picture playback apparatus ofEmbodiment 2.

This embodiment represents a case in which moving pictures are playedback or edited without being digitally compressed and read into acomputer, and moving pictures which are the edited result, are obtained,for instance, by dubbing the moving pictures from one VTR to anotherVTR. Therefore, this configuration requires two or more sequentialaccess type moving picture storage units.

(1) Defining the Scene

The scene definition method is basically the same as the conventionalnon-linear edit system operation.

(2) Reading the Moving Pictures

A typical frame determination unit 21 selects one or more typical framesfor each scene defined by scene definition unit 20. There are severalmethods for selecting typical frames as follows:

The most typical method is to select the top frame of each scene as thetypical frame.

Or, it is possible to select typical frames at a certain interval, suchas a first frame, 31st frame, 61st frame, and so on for each scene.

Or, it is also possible to select frames having a remarkable change bylooking into the moving pictures of each scene as the typical frame.

A typical frame storage unit 22 stores the moving picture data of theframe determined by typical frame determination unit 21. At this time,moving picture data may be stored in different files for each frame, ortypical frames included in the scene may be stored in one filecollectively. Also, frames may be stored in a bitmap format withoutbeing compressed, or compressed in the JPEG format and so forth.

The frame number and the moving picture data of the frame are stored asa set in typical frame storage unit 22. The obtainment of the picturedata to be stored is performed as explained below. When the definitionof a scene is finished by scene definition unit 20, typical framedetermination unit 21 performs determination of a typical frame for eachscene and obtainment of picture data of the typical frame. In order toobtain the picture data, typical frame determination unit 21 positionssequential access type moving picture storage unit 2 to each typicalframe, and the frame is read there and stored in typical frame storageunit 22.

(3) Displaying the Scene

A moving picture display unit 9 plays back the moving pictures of eachscene according to the instruction from the user and displays them onthe screen. Sequential access type moving picture storage unit 2 canplay back stored moving pictures sequentially from the start, but if itattempts to play back the scene defined by scene definition unit 20, thescene cannot be immediately played back because the video tape, etc.,must be forwarded to the position of the scene. Or, when a new scene isdefined by concatenating two scenes, if an attempt is made to playbackthe concatenated scene, after one scene is played back, the playback isdisconnected until the video tape, etc., is forwarded to the next scene.

Therefore, in this embodiment, the typical frame of each scene is readfrom typical frame storage unit 22 and displayed. The picture data ofthe selected typical frames is stored in typical frame storage unit 22,and moving picture display unit 9 reads the picture data of the storedtypical frame in accordance with the timing of the frame number anddisplays. When playback is performed in this way, it is possible toconfirm the rough content although a smooth playback is not.

When the apparatus is configured in this way, although it is sufficientto playback and confirm the content roughly, it is not suited to fineediting work because a frame cannot be displayed unless the frame is atypical frame when a specific frame is to be confirmed. Therefore, it ispossible to configure the apparatus in such a way that when the frameforward, frame rewind, stop, and other display command are input, if theframe to be displayed is not the typical frame, sequential access typemoving picture storage unit 2 positions the medium to the frame so as toread the picture data and display.

The operational flow of moving picture display unit 9 of this embodimentis illustrated in FIG. 4.

The processing procedure of moving picture display unit 9 of FIG. 4 isexplained.

First, processing starts at step S10, and the starting frame number ofthe scene is substituted as the current frame number FT at step S11. Thedisplay processing is performed for the frame number FT at step S12.Then, an event is waited at step S13. After a time for one frame haspassed, a determination is made whether moving pictures are being playedback or not at step S14, and if they are being played back, the currentframe number FT is forwarded by one at step S15, and the next frame isdisplayed at step S22. On the other hand, if the moving pictures are notbeing played back, the procedure returns to step S13 and again waits forthe event.

When the playback button is pressed while waiting for the event at stepS13, a playback flag is set at step S16, and the display processing ofthe frame number FT is performed at step S22.

When the stop button is pressed, the playback button is cleared at stepS17, and display processing is performed for the frame number FT at stepS22. When the frame forward button is pressed , the playback flag iscleared at step S18, the current frame number FT is forwarded by one atstep S19, and the next frame is displayed at step S22.

Furthermore, when the frame rewind button is pressed, the playbackbutton is cleared at step S20, the current frame number FT is returnedby one at step S21, and the previous frame is displayed at step S22.

The flow of frame display processing of moving picture display unit 9 isshown in FIG. 5.

The frame display processing procedure of FIG. 5 is explained.

At step S30, whether or not the frame of the current frame number FT isa typical frame is determined. If it is a typical frame, the currentframe number FT is substituted as the frame number FN to be displayed.At step S32, the frame number FN to be displayed is confirmed if it isthe same as the currently displayed frame number FC, and if it is not,the picture data of the frame of the frame number FN is read fromtypical frame storage unit 22 and displayed on moving picture displayunit 9 at step S 33.

When it is not a typical frame at step 30, playing back or not is judgedat step S35. When it is not being played back, the current frame numberFT is substituted as the frame number FN to be displayed at step S36. Atstep S37, the frame number FN to be displayed is confirmed to be thesame or not as the currently displayed frame number FC, and if not, thepicture image data of the frame number FN to be displayed is read fromsequential access type moving picture storage unit 2 at step S38, anddisplayed on moving picture display unit 9.

When being played back at step S35, the number of the closest typicalframe before the current frame number FT is substituted for the framenumber FN at step S40. AT step S41, the frame number FN to be displayedis confirmed whether it is the same or not as the currently displayedframe number FC, and if not, the picture data of the frame of the framenumber FN is read from typical frame storage unit 22 at step S42, anddisplayed on moving picture display unit 9.

When the stop, frame forward, or frame rewind button is pressed,processing is executed in the order of step S36, step S37, step S38 ifit is not the typical frame, and if it is the typical frame, theprocessing is executed in the order of step S31, step S32, step S33.

When moving picture display unit 9 is configured in this manner, onlyselected typical frames are read in computer 23 memory, that is, typicalframe storage unit 22 to facilitate the playing back and editing work.

(4) Executing Editing Operation

The operation of scene processing, concatenating, or compositing forthis embodiment are basically the same as those of conventionalnon-linear editing system.

(5) Creating Moving Pictures

The new scene information created as the result of editing is stored inscene information storage unit 5. Based on the scene information, movingpicture creation unit 11 creates moving pictures which are the editingresult on the video tape, etc., of sequential access type moving picturestorage unit 2.

The scene information of the moving pictures to be output is processed,concatenated, and compositited based on those scenes to be input. Movingpicture creation unit 11 inputs one or more scenes stored in sequentialaccess type moving picture storage unit 2, and then outputs editedscenes to another sequential access type moving picture storage unit 2.For compositing two scenes, these scenes are to be stored in separatesequential access type moving picture storage unit 2. Moving picturecreation unit 11 reads scenes in the order of time axis defined by thescene information and plays them back. If processing is demanded for thescene information, the result of processing based on the demand isoutput. Also, if composition is demanded, the scene is played back bytwo sequential access type moving picture storage unit 2 respectively,and then composited result is output. In this way, the moving picturesof edited scenes are created in sequential access type moving picturestorage unit 2 for output.

According to the embodiment, as thus far described, because editing canbe performed without compressing the moving pictures digitally andreading them into the computer memory, the time required for digitallycompressing moving pictures can be saved.

Moreover, an extension board for compressing moving pictures digitallyor a large capacity memory for storing digitally compressed data are nolonger required.

Embodiment 3

According to Embodiment 2, editing can be performed without digitallycompressing moving pictures into data and storing the data in a computermemory. However, in Embodiment 2, because the moving pictures as theedited result are obtained by dubbing the moving pictures from one VTRto another VTR, two or more VTRs must be connected to the computer. Onthe other hand, according to Embodiment 1, although one VTR can beconnected to the computer because digitally compressed moving picturedata can be stored in the computer memory, a large capacity memory hasto be incorporated in the computer.

In addition to the editing computer, a server computer provided with alarge capacity memory is attached, and the digitally compressed movingpicture data is stored in the server computer, and by using the movingpicture data, a configuration which enables playing back and editingmoving pictures by the apparatus not equipped with a digital movingpicture compression unit or moving picture storage unit is outlinedbelow.

FIG. 6 shows a configuration of a moving picture playback apparatus ofEmbodiment 3.

The moving picture playback apparatus is configured by a server computer30 connected to the communication network and an editing computer 32.Server computer 30 is connected to sequential access type moving picturestorage unit 2 such as a VTR.

Server computer 30 includes a moving picture digital compression unit 6,a direct access type moving picture storage unit 7 and a moving picturecreation unit 11. Also, editing computer 32 includes a scene definitionunit 20, scene information storage unit 5, a typical frame determinationunit 31, a typical frame storage unit 22, a command input unit 8, amoving picture display unit 9, and an editing operation execution unit10.

The operation of this Embodiment is set forth below.

(1) Defining a Scene

The definition method of the scene is basically the same as theoperation of the conventional non-linear editing system.

For defining a scene, firstly, a medium such as a video tape whichrecorded moving pictures is set to the sequential access type movingpicture storage unit connected to server computer 30. Although it issupposed that server computer 30 is geographically far from editingcomputer 32, because moving pictures can be carried by the medium suchas the video tape, a unit such as mailing can be selected.

(2) Reading Moving Pictures

For reading moving pictures, the moving pictures are read fromsequential access type storage unit 2 by moving picture digitalcompression unit 6, and then digitally compressed, and the compresseddata is stored in direct access type moving picture storage unit 7.Typical frame determination unit 31 selects a typical frame, and typicalframe storage unit 22 reads a moving picture data of the typical frameamong the digitally compressed moving picture data stored in directaccess type moving picture storage unit 7 of server computer 30 via thecommunication network and stores the data.

(3) Displaying a Scene

Moving picture display unit 9 displays scenes by using the typical framestored in typical frame storage unit 22 of editing computer 32 and themoving pictures stored in direct access type moving pictures storageunit 7 of server computer 30. That is, display is conducted by usingtypical frames during playback, and when frames other than typicalframes must be correctly displayed at the time of frame forward, framerewind, or stop, the data of the frame corresponding to the moving datais read via the communication network, and the frame is displayed.

(4) Executing the Editing Operation

This is basically the same as the operation of the conventionalnon-linear editing system.

(5) Creating Moving Pictures

As the edited result, a new scene for output is defined, and the sceneinformation is stored in scene information storage unit 5. The sceneinformation is transmitted to server computer 30 via the communicationnetwork. At server computer 30, moving picture creation unit 11 createsmoving pictures for output based on the scene information and the movingpicture data stored in direct access type moving picture storage unit 7.The operation of moving picture creation unit 11 is basically the sameas the operation of moving picture creation unit 11 of Embodiment 1.

In this Embodiment, because the moving picture data is transmitted toediting computer 32 as necessary by the communication unit by providingmoving picture digital compression unit 6 and direct access type movingpicture storage unit 7 to server computer 30, an extension board or alarge capacity memory is no longer needed for moving picture digitalcompression for editing computer 32.

Embodiment 4

The configuration to be next described is that a typical frame isdetermined based on an event detected by the event detection unit, andthe frame is immediately read and stored so as to start edit operationas soon as the shooting ends.

FIG. 7 shows a configuration of moving picture playback apparatus ofEmbodiment 4.

In this configuration, a video camera 1 attached with a sequentialaccess type moving picture storage unit 2 and an event detection unit 3,a server computer 30 and an edit computer 41 are outlined.

The operation of this embodiment is outlined below.

(0) Shooting

Similarly to the operation of shooting of Embodiment 1, event detectionunit 3 detects a certain event during shooting and notifies editcomputer 41 of the detected event.

(1) Defining a Scene

Similarly to Embodiment 1, when event detection unit 3 detects an event,scene definition unit 4 immediately defines a new scene information withthe frame corresponding to the event delimiter, and stores it in sceneinformation storage unit 5.

(2) Reading Moving Pictures

For reading moving pictures, the moving pictures are read by movingpicture digital compression unit 6 from sequential access type movingpicture storage unit 2 and digitally compressed, and the compressed datais stored in direct access type moving picture storage unit 7. Also,typical frame determination unit 40 selects a typical frame according tothe event notified by event detection unit 3 during shooting.

After the typical frame is selected, typical frame storage unit 22immediately reads the frame from sequential access type moving picturestorage unit 2 of video camera 1, and stores.

(3) Displaying a Scene

The operation is basically the same as the scene displaying ofEmbodiment 3. However, it is possible to display scenes using sequentialaccess type moving picture storage unit 2 similarly to Embodiment 2,until the reading of previously mentioned moving pictures is completed.

(4) Executing the Edit Operation

The operation is basically the same as the creation of moving picturesof Embodiment 3.

According to this Embodiment, event detection unit 3 detects an eventduring shooting, and scene definition unit defines the scene at the sametime, and typical frame storage unit 22 reads the typical frame fromvideo camera 1 and stores, therefore, the time for defining the sceneand reading moving pictures can be saved, and edit operation can beimmediately started as soon as the shooting ends.

Embodiment 5

A configuration of a moving picture collection apparatus is nowexplained. Based on an event detected by event detection unit 3, theframe number and moving picture data at the time of event occurrence areobtained, and the event is specified by storing them in eventinformation storage unit 64, and the contents of the moving picturescorresponding to the event can be quickly confirmed

FIG. 8 shows a configuration of the moving picture collecting apparatusof Embodiment 5.

The moving picture collection apparatus of this embodiment is configuredby a video camera recorder 60 attaching sequential access type movingpicture storage unit 2 and event detection unit 3 to video camera 1 anda computer 61 connected to it, similarly to the moving picture playbackapparatus of Embodiment 1. Computer 61 is equipped with a moving pictureobtainment unit 62 for obtaining a moving picture at an arbitrary pointfrom video camera recorder 60, a frame number obtainment unit 63 forobtaining the frame number showing the position on the video tape whichis being played back or recorded at that time by video camera recorder60, and an event information storage unit 64 for storing the framenumber obtained by frame number obtainment unit 63 or the moving picturedata obtained by moving picture obtainment unit 62 by correlating withthe event detected by event detection unit 3.

Furthermore, computer 61 is provided with al display unit 65 fordisplaying the event information stored in event information storageunit 64, a command input unit 66 for the user (a person using thismoving picture collection apparatus) to input commands such as an eventselection end instruction, and a playback instruction unit 67 forspecifying the frame number to video camera recorder 60 and instructingto playback from the position on the video tape corresponding to theframe number.

The operation of this embodiment is set forth below.

The user performs shooting using video camera recorder 60.

At this time, event detection unit 3 detects the change of shootingsubject, the change of video camera 1, or the event of user operation,and notifies computer 61 of the type of the event. The operation forevent detection unit 3 to detect events is the same as that ofEmbodiment 1.

When an event is detected by event detection unit 3, the eventoccurrence and the type of the event are notified computer 61 from videocamera recorder 60. Upon receiving the notification of event occurrence,computer 61 immediately obtains the frame number at that time from videocamera recorder 60 by frame number obtainment unit 63, and obtains themoving picture data at that time by moving picture obtainment unit 62.These frame numbers and the moving picture data are correlated with theevent type and stored in event information storage unit 64.

FIG. 9 explains the event information stored in event informationstorage unit 64. In this figure, events of recording start and recordingend, which are the operation of the user during shooting, are correlatedwith the frame numbers and stored. Furthermore, the moving picture data,at that time is stored for the event at the recording start.

When the shooting ends, the user confirms the contents of the shotmoving pictures.

Computer 61 creates an index based on the event information stored inevent information storage unit 64, and displays it on display unit 65.

FIG. 10 explains the index to be displayed on display unit 65. In thefigure, a frame corresponding to the event at the shooting start isselected as the element of the index. The moving picture datacorresponding to the elements of each index are aligned and displayed asthe index.

By looking at the index, the user selects a desired element by clickingthe position, for instance, on which the moving picture data isdisplayed. When the element is selected, the frame number correspondingto the element can be determined by referring to event informationstorage unit 64, playback instruction unit 67 notifies video camerarecorder 60 of the frame number, and instructs to playback from theposition of this frame number.

When the frame number is notified from computer 61, and an instructionis given to playback from that position, video camera recorder 60instructs sequential access type moving picture storage unit 2 to rewindor forward up to the position of the specified frame number. As thepositioning to the specified frame number position is ended, a nextinstruction is given to sequential access type moving picture storageunit 2 to start playback.

Based on the events collected by the moving picture collectionapparatus, the frame number and the moving picture data at the eventoccurrence are obtained, and stored in event information storage unit 64attached to the moving picture collection apparatus so as to specify theevent and quickly confirm the content of the moving picturescorresponding to the event. However, it is possible to send theinformation stored in event information storage unit 64 to the movingpicture playback apparatus of Embodiments 1 to 4 and perform movingpicture edit processing such as playback, concatenation, or compositionof moving pictures by this moving picture playback apparatus.

For sending the information stored in event information storage unit 64to the moving picture playback apparatus of the previous Embodiments 1to 4, the previously mentioned moving picture collection apparatus andthe moving picture playback apparatus can be connected by communication,or the information stored in event information storage unit 64 can bestored in a recording medium such as floppy disks and that recordingmedium can be set to the moving picture playback apparatus for use.

With referring to the flowcharts of FIGS. 11 and 12, the procedure ofoperation processing from event detection to the playback of the movingpictures is explained.

FIG. 11 is a flowchart showing the processing flow from obtaining themoving picture data and the frame number at the time of event occurrencebased on the event detected by event detection unit 3, and then storingin event information storage unit 64.

Firstly, at step S51, the user starts shooting using video camerarecorder 60, and the event occurrence is checked at step S52. When theevent is detected, the frame number and the moving picture data at thetime of event occurrence are obtained at step S53.

Then, the frame number and the moving picture data are stored in eventinformation storage unit 64 at step S54, and the occurrence of a nextevent is checked again at step S52. When no event is detected, shootingend or not is checked at step S55, and if it is not a shooting end,event occurrence is again checked at step S52, and the operation ends ifit is the shooting end.

FIG. 12 is a flowchart showing the processing flow of confirming thecontents of the moving pictures the user shot.

At step S56, the index created based on the frame number and the movingpicture data stored in event information storage unit 64 is displayed ondisplay unit 65, and whether or not a command is input is checked atstep S57. When no command has been input, the command input confirmationis checked until a command is input. When a command is input, whether ornot the element of the index which is the frame corresponding to theevent is checked at step S58.

When the index element is selected, an instruction to playback from theframe number corresponding to the element selected by playbackinstruction unit 67 is issued at step S59, and after that, whether ornot a next command is input is checked again at step S57. When the indexelement has not been selected, whether or not the end is instructed ischecked at step S60, and if ending is not instructed, a command input isagain checked at step S57, and the operation ends when the end isinstructed.

According to the Embodiment, when an event is detected by eventdetection unit 3, frame number obtainment unit 63 immediately obtainsthe frame number at that time, and event information storage unit 64stores the frame number correlating with the event information.Therefore, the user can playback the moving picture at the time of eventoccurrence and confirm by referring to the index and selecting thespecific event.

Embodiment 6

A configuration of having a unit with which the user can add explanationsentences to each event to Embodiment 5.

FIG. 13 is a configuration showing the configuration of a moving picturecollection apparatus of Embodiment 6. To the configuration of Embodiment5, an explanation sentence input unit 68 is added to this Embodiment.

Explanation sentence input unit 68 inputs an explanation sentence to theevent stored in event information storage unit 64. A text input frameallocated for each moving picture corresponding to each event isdisplayed. The user selects this and inputs an explanation sentence fromthe keyboard, and the event information attached with the inputexplanation sentence is stored in event information storage unit 64.

FIG. 14 explains the event information stored in this event informationstorage unit 64.

FIG. 15 explains the index displayed on display unit 65.

The user inputs the explanation sentence to each of the indexesdisplayed on display unit 65. In the example of this figure, theexplanation sentence of “a scene of the airport” to the first movingpicture, and the explanation sentence of “a scene of the railroad” isinput to the second moving picture.

By adding and storing the explanation sentences to each event in eventinformation storage unit 64 in this way, when the index is displayednext time, the index with the explanation sentence is to be displayed.

According to this Embodiment, because the explanation sentence the userinput can be stored in event information storage unit 64 correlatingwith the event, the contents of the moving pictures can be confirmedwhen the shooting has finished, and further, the explanation sentencefor a specific frame of each event, e.g., in the moving pictures can beadded. In performing edit operation, etc., for these moving picturelater on, the index with attached explanation sentences can be displayedto see, therefore, a necessary frame can be quickly found by referringto the explanation sentences.

Embodiment 7

FIG. 16 shows the configuration of the moving picture collectionapparatus of Embodiment 7. This embodiment assumes that the movingpicture collection apparatus is attached to a vehicle, and movingpicture information can be collected by performing the shooting whilerunning in the streets.

In this Embodiment, a position detection unit 70 such as the GPS (GlobalPositioning System) is connected to a RS232C port, etc., of thecomputer, so as to obtain the longitude and latitude information of theapparatus at any given point, on top of the configuration of Embodiment5. It is also provided with a setting input unit 71 for settingbuildings or area to be shot beforehand. And it has a control unit 72for controlling, etc., the starting, ending the shooting and thedirection of the camera.

The operation of this embodiment will be next explained.

With set input unit 71, the user inputs the spots or area to be shot, orthe type of geographical elements to be registered as the index prior tothe shooting.

FIG. 17 shows an example of the set input unit 71 screen.

In this example, the longitude, latitude of the shooting start point,and the longitude and latitude of the shooting end point are set. Thesetting of the longitude and latitude can be directly input with anumeric value, or the numeric values of the longitude and latitudecorresponding to the spot can be input by clicking the shooting startpoint and shooting end point on the map window.

Or, instead of specifying two points of shooting start point andshooting end point, the range of area to be shot can be specified. Forinstance, a method of specifying an area corresponding to the shootingarea by a polygon on the map window, and inputting the longitude andlatitude of the apices positions of the polygon can be thought of.

Furthermore, a method of inputting the names of specific buildings, suchas ×× bldg. or ΔΔ shrine as the shooting subjects can be thought of. Insuch a case, a database describing the longitude and latitude of eachbuilding is prepared, and the longitude and latitude of the buildingswhereof the names have been input can be searched from the database andset.

The index is explained next. Because many buildings are recorded whenmoving pictures are collected for a long time, and when the user wantsto find out a moving picture having a certain building, it is not easyto find out the moving picture having the certain building later on.Therefore, a building characteristics of each region, for instance,scenes taking ten or more storied buildings only are separated andregistered as the index, then, if a scene of a tall building close tothe shooting subject building can be found using the index, it isassumed that the shooting subject scene is located in vicinity. Theconditions for geographical elements to be registered as the index arespecified by set input unit 71. For instance, when a condition such as“10 or more storied buildings” is input, the geographical elementsatisfying the condition within the shot area is selected by searchingthe database for each geographical element. The longitude and latitudeinformation selected in this way can be set.

The operation for the shooting is explained next. For shooting, videocamera 1 is attached to the roof or the side of the vehicle, forinstance, and computer 76 is set in the vehicle, and they are connectedwith a cable. It is configured in such a way that video camera 1 cancontrol the shooting start or end by sending a control signal fromcomputer 76 via the cable. Video camera 1 is attached to an electricbase so that its direction can be controlled from the instruction ofcomputer 76.

Position detection unit 70 detects the longitude and latitudeinformation of the current position of the vehicle periodically, forinstance, at every minute. Event detection unit 3 compares the longitudeand latitude information of the current position of the vehicle detectedby position detection unit 70 with each position information set by setinput unit 71, and when a certain event is detected, reports the eventoccurrence and the type of the event. The events to be detected are;approach of the vehicle to the shooting start, end positions, entry ofvehicle into the shooting area, leaving of the vehicle from the shootingarea, an approach of the vehicle to the building which is the shootingsubject, an approach of the vehicle to the building which is to beregistered as the index, and so on.

When an event of approach to the shooting start position or entry intothe shooting area is reported, control unit 72 sends a command forshooting start to video camera 1. When an event of approach to theshooting end point or the leaving from the shooting area is reported,control unit sends a command of shooting end to video camera 1. In thisway, the area automatically set can be shot without the user'sperforming the operation of shooting start or end while moving in thevehicle.

When a building to be shot is specified, event detection unit 3 detectsthe approach of the vehicle to the building assumed to be the shootingsubject as an event and reports. Upon receiving the report of thisevent, control unit 72 sends a command of shooting start to video camera1, and controls video camera to be directed toward the building assumedto be the shooting subject.

The direction of the video camera 1 can be controlled in the followingway. When the event of approaching to the specific building is reportedfrom event detection unit 3, control unit 72 obtains the longitude andlatitude information of the building based on the information set by setinput unit 71. Then the information of the current vehicle position isobtained by position detection unit 70. Furthermore, using the jairosensor, etc., the direction of the vehicle is also obtained. Concerningthe information of the vehicle direction, there is a method of decidingthe direction toward which the vehicle is to proceed by analyzing thechange of the vehicle position for the past few seconds, and estimatingthe direction as the direction of the vehicle.

When the position of the building to be shot and the position anddirection of the vehicle are determined, the direction of video camera 1to be set is determined based on them. Based on the determineddirection, control unit 72 controls the base on which video camera 1 isattached and changes the direction of video camera 1, so that theshooting subject comes into the area of shooting for video camera 1.

When the building to be registered as the index is specified, eventdetection unit 3 detects an approach of the vehicle to the building tobe registered as the index as the event and reports. When this event isreported, control unit 72 controls the direction of video camera 1 sothat the building comes into the shooting area, and moving pictureobtainment unit 62 obtains the moving picture data at that time andstores in event information storage unit 64 with the event information.

Examples of event information stored in event information storage unit64 by the above mentioned operation is shown in FIG. 18.

The operation of apparatus for the user to confirm the shot contentafter the shooting ends is explained next. On display unit 65, the indexas shown in FIG. 19 is displayed based on the event information storedin event information storage unit 64. When a moving picturecorresponding to one event among the displayed list is clicked, videocamera recorder 75 is controlled and the corresponding moving picture isplayed back. This is the same as for Embodiment 5.

Embodiment 8

The moving picture collection apparatus for this Embodiment isconfigured by a moving picture input unit such as a video camera, etc.,an event detection unit for detecting events, and an event informationstorage unit which obtains the position information of the shootingapparatus by GPS devise, etc., corresponding to the event detected bythis event detection unit and expresses the obtained information by themoving picture or sound, and composites it with the moving picture inputfrom the moving picture input unit and then stores on a video tape andso on.

FIG. 20 explains one embodiment example of a position informationattachment filter of Embodiment 8, and FIG. 21 shows the frontappearance of the position information attachment filter.

The event information storage unit performs composition of the eventinformation and the moving pictures using position informationattachment filter 80 shown in FIGS. 20 and 21. The position informationattachment filter is cylindrical, and used by attaching it to the frontof the lens of video camera 1. Position information attachment filter 80is made of a transparent body, and a display area 83 in which characterstrings can be displayed is provided in part of the transparent body anda GPS receiving connector 82 for receiving signals from the GPS deviceand so forth.

Being configured in this way, when an event such as a shooting start ormoving of collection apparatus is detected by event detection unit 3,position information attachment filter 80 displays the character stringsindicating the position information in the transparent specific areacorresponding to the event.

The operation of this embodiment will be next explained.

Position information attachment filter 80 reads the longitude andlatitude information received by the GPS device periodically, forinstance, at every minute, via GPS receiving connector 82. When theevent of shooting start is detected, position information attachmentfilter 80 converts the longitude and latitude information into characterstrings and displays them on the transparent specific area. The positioninformation is recorded on the video tape together with the pictures.

After the shooting apparatus has moved, and a gap larger than thepredetermined is generated between the currently displayed positioninformation and the position information received from the GPS device,event detection unit 3 reports the event of the shooting apparatus move.When the event of the shooting apparatus move is reported, positioninformation attachment filter 80 updates the display of the positioninformation corresponding to the event.

When the video tape recorded in this way is played back using the VTR,the position information for the shot place is always displayed on thescreen. When the forward or rewind function of the VTR is used, a scenewith the shot subject point can be quickly found. In the above example,an embodiment of embedding the longitude and latitude information of theshot position in the shot moving picture in character strings isexplained, but the longitude or latitude information of the shotposition can be converted into the name of the place information, etc.,using the geographical information database and then the name of theplace can be embedded into the moving picture as character strings.

The above example shows a case in which character strings to be embeddedand shot moving pictures are optically composited at the same time asthey are shot using a transparent display unit to be recorded on a videotape, it is also possible to record the position information to beembedded and shot moving pictures separately, and to embed the characterstrings into the moving pictures after the shooting has ended.

Further more, it is obvious that the display of position information isnot limited to character strings, a map can be displayed graphically,and symbols can be placed to the corresponding position in the map toindicate the position, or position information can be converted intosound by audio composition to be embedded into the sound track of thevideo tape, or the similar effect can be obtained by using graphics,symbols, sound, and so on.

Embodiment 9

A moving picture playback apparatus equipped with a playback speed setunit which can playback the moving pictures at an arbitrary playbackspeed and then confirm the contents of the moving pictures according tothe playback speed quickly will be next explained.

FIG. 22 shows a configuration of the moving picture playback apparatusof Embodiment 9.

The moving picture playback apparatus of this embodiment is providedwith a frame obtainment unit 309 for obtaining images of the frames ofthe moving pictures, a priority obtainment unit 305 for obtaining thepriority set in the frames of the moving pictures, a playback speed setunit 301 for setting the playback speed, a frame selection unit 303 fordiscarding/selecting frames of the moving pictures based on the playbackspeed set by playback speed set unit 301 and by the priority of theframes of the moving pictures obtained by priority obtainment unit 305,and a moving picture display unit 307 for displaying the images of theframes selected by frame selection unit 303 and obtained by frameobtainment unit 309. The priority of the moving picture frames is set bypriority set unit 297, and the set priority is memorized in a prioritystorage unit 299. A computer 310 is provided with them.

Frame obtainment unit 309, for instance, reads the digital datacompressed by the moving picture compression system such as MPEG (MotionPicture Experts Groups), and obtains the image data of each frame.

Priority obtainment unit 309 obtains the priority set for each frame.The priority information is set for each frame by priority set unit 297,and memorized in priority storage unit 299 which memorizes priority.

FIG. 23 shows a priority table for memorizing the priority set for eachframe of the moving pictures, as one example of the priority storageunit.

In this example, priority table 330 memorizes frame numbers 321 foridentifying each frame and priority 323 set for the frames incorrelation. In this example, the priority is expressed with numbers,and priority 1 indicates the top priority and the priority becomes lowerin the order of priority 2, and priority 3. It is possible to expresswith letters such as A, B, C, and so on, instead of the numbers. In thisexample, the top priority 1 is set for the frames with frame numbers n1,n2, n3, and n4. Frame numbers n21, n22, n41, n42 are set as priority 2,and frame number n211 and frame number n421 are set as priority 3. Inthis example it is assumed that other frames not registered on thepriority table are regarded as the lowest priority 4. It is alsopossible to incorporate a priority table which sets priorities not onlyfor frames with higher priority but for all frames.

With referring to FIG. 24 and FIG. 25, the operation flow of priorityset unit 297 is explained.

FIG. 24 shows the operation flow of the priority set unit. FIG. 25 showsan example of display screen for setting the priority according topriority set unit 297. A priority set screen 511 is also indicated.

First of all, the moving picture data is read and its leading frame isdisplayed (S401) on a moving picture display window 513 on the right ofpriority set screen 511. Then, at S403, a command is input. At S405, aprocessing corresponding to the input command is performed. First, usinga control button such as playback button 515, stop button 516, framerewind button 517, frame forward button 518, etc., on the bottom ofmoving picture display window 513, a frame the priority of which to beset is selected and positioned as the current frame (S407). Then, when a“frame register” button 514 is pressed, the frame currently displayed onmoving picture window 513 is registered in a frame list 520 on the leftscreen. The registered frame becomes the frame in selected condition. Onframe list 520, frames with higher priority are displayed on the left,and frames with lower priority are displayed on the right. Priority 1,which is the top priority is first set for the frame added to frame list520, and the images corresponding to the frames are displayed (S417) onthe leftmost side of frame list 520, as indicated at 521, 523, and 531.When a “Priority Down” button is pressed, the priority of the selectedframe downs, and the position of the frame moves rightwards (S409) as525 on frame list 520 accordingly. Similarly, when a “Priority Up”button 545 is pressed, the priority ups, and the position of the framemoves leftwards (S411) on frame list 520 accordingly. In order to changethe frame in the selected condition, frame selection is performed (S413)by clicking a frame on the frame list. Frame selection is the operationwhich makes a frame registered in frame list 520 to the selectedcondition. When a “frame delete” button 543 is pressed, the frame in theselected condition currently is deleted from the frame list. The deletedframe is deleted from priority table 330 when the setting ends.

When the display is made using frame list 520 in this way, the priorityset for each frame can be confirmed visually by the image positiondisplayed in the frame list.

When the priority setting is complete, the result is stored in the file(S419) as the priority table as shown in FIG. 23. Although buttons to beused for ending the setting are not illustrated in FIG. 25, the prioritysetting is to be completed by such operations as the closing of thewindow which displays this screen, or by the predetermined operationsprovided by the system.

Frames with higher priority mean the frames to be displayed withoutbeing omitted during forwarding or rewinding, and frames with lowerpriority mean frames which can be omitted during forwarding orrewinding.

Setting the priority to be high or low is described below.

As an example of setting the priority high or low suited to the movingpicture playback apparatus of this embodiment, there is a method ofdefining a scene for the moving picture, and setting a high priority forthe first frame of each scene.

The case of applying this method is explained in concrete below.

FIG. 26 indicates the relations between the structure of the scenes andthe priority set for the frames. The relations between the structure ofthe scenes and the priority set for the frames indicated in FIG. 26corresponds to the contents of the priority table shown in FIG. 23. Incase of this example, the moving picture is largely divided into fourmajor scenes from scene 1 to scene 4, based on the meanings of thecontents. Of them, scene 2 and scene 4 are divided into three mediumscenes (scene 2-0 to scene 2-2, scene 4-0 to scene 4-2). Further, scene2-1 and scene 4-2 are divided into two small scenes (scene 2-1-0, scene2-1-1, scene 4-2-0, and scene 4-2-1). The leading frame of the majorscene is set to priority 1, the leading frame of the medium scene butnot the leading frame of the major scene is set to priority 2, and theleading frame of the small scene but not the leading frame of the mediumscene is set to priority 3.

Setting priority this way enables frame selection according to the setpriority. As a result, the leading frames n1, n2, n3, and n4 of themajor scenes can be displayed at the high-speed forwarding, and theleading frames n21, n22, n41, and n42 of the medium scenes can bedisplayed at the mid-speed forwarding, and the leading frames n211, n421of the small scenes can be displayed at the low-speed forwarding. Alsoit is possible that the leading frames n1, n2, n21, n22, n3, n4, n41,and n42 of the major scenes and medium scenes are displayed at themid-speed forwarding, and the leading frames n1, n2, n21, n211, n22, n3,n4, n41, n42, n421 are displayed at the low-speed forwarding.

By selecting the leading frame of each scene, an example of setting highpriority is shown in the previous example. As other examples, it is alsopossible that high priority can be set by selecting frames other thanthe leading frame, or high priority can be set by selecting not just oneframe but two or more frames for each scene. In either method of settingpriority, the display reflecting the priority set by the user can berealized at forwarding in an unchanged manner.

Now, the explanation is given about playback speed set unit 301, butprior to the explanation, the normal playback speed is explained.

Unlike still pictures, moving pictures are the data obtained as timepasses by. Namely, moving pictures are collected in real time. Thecollected moving picture data holds the time information which indicatesthe time in addition to the moving picture information. The normalplayback speed in the explanation hereinbelow refers to the playback inthe time the same as the time passed during the collected movingpictures.

Playback speed set unit 301 sets the speed at which the moving picturesare played back. Examples of playback speed set screen displayed byplayback speed set unit 301 are shown in FIG. 27 through FIG. 29.

In the examples shown in FIG. 27 and FIG. 28, the playback speed can bechanged by sliding handles 551, and 561 from left to right,respectively, with slid bars 550 and 560 displayed on the screen. Whenhandles 551 and 561 are on the right side from the center, it means theplayback in the forward direction, and when they are on the left sidefrom the center, it means the playback in the reverse direction. Whenhandles 551 and 561 are in the center, it means still, and when handles551 and 561 are on the edge, it means the high-speed forwarding (on theright edge) or rewinding (on the left edge), respectively. The number ontop of slide bar 550 in FIG. 27 indicates the magnification (the ratiofor the playback time of the real time of the moving picture data) forthe normal playback speed. For instance, 3 means the playback isperformed at the speed three times faster than the normal speed. Thatis, the playback is performed in the time one third of the time passedin the moving pictures.

FIG. 28 shows an example of setting the playback speed in each stage,“High-Speed Forwarding”, “Low-Speed Forwarding” and “Normal Playback”and so on.

FIG. 29 shows an example in which slide bar 570 is corresponding to theleading frame to the last frame of the moving picture. The position ofthe frame to be displayed is specified by sliding handle 571. And theplayback speed is specified by the speed at which handle 571 is sled.Sliding handle 571 at high speed specifies the high-speed playback, andsliding handle 571 at low speed specifies the low-speed playback.

Frame selection unit 303 discards/selects frames of the moving picture.Frame obtainment unit 309 can obtain the image of the frame specified atthe speed of 5 frames per second, in this embodiment. Let us assume thatthe playback speed is set with a magnifying speed of 10. And if a movingpicture is composed of 30 frames per second, the playback with amagnifying speed of ten corresponds to the playback display of themoving pictures of 300 frames per second. Given the processing capacityof the previously described frame obtainment unit 309, however, frameselection unit 303 is to display by selecting 5 frames among 300 frames.

In this case, the simplest method of frame selection is to select framesat an equal interval. Namely, if the frame numbers of the 300 frames tobe played back are m, m+1, . . . m+299, then, frames m, m+60, m+120,m+180, m+240 can be selected. When the frame is selected by frameselection unit 303 in this way, frame obtainment unit obtains the imageof the selected frame, and moving picture display unit 307 displays theobtained frame image.

According to the above method of selecting frames in an equal interval,forwarding or rewinding at the set playback speed is made possible, butthe method poses a concern that an important scene may be skipped andnot displayed. For instance, a scene of Mt. Fuji is inserted between theframe number m+10 and the frame number m+50, and the user is searchingfor the scene of Mt. Fuji by forwarding with the magnifying speed of 10,not a single frame constituting the scenes of Mt. Fuji is going to bedisplayed in the above method. Therefore, the priority is set as shownin FIG. 23, based on the setting method of priority previouslydescribed. That is, scenes are defined based on the content of themoving pictures, and a high priority is set by selecting frames for eachdefined scene. In this way, it is guaranteed that at least one frame isselected for a short scene and diploid during forwarding As a result, atthe time of forwarding, the display based on the contents of the movingpicture can be realized.

FIG. 30 to FIG. 32 show examples of operation flows of the movingpicture playback apparatus of the above embodiment. Hereinafter, theexplanation is given according to the flows.

The codes used for FIG. 30 to FIG. 32 are explained first. F indicatesthe current frame position, F0 is the original frame position, f is thenumber of frames per second, g is the number of frames displayed persecond during forwarding or rewinding, and m indicates the set playbackspeed.

The explanation is given in accordance with FIG. 30. At S431, 1 is setfor the current frame position F and for the original frame position F0.Then, at S435, the set playback speed m is obtained. The absolute valueof the obtained playback speed m is determined whether it exceeds “1”(S437), and then either the normal playback, frame forward, frame rewindprocessing at S440 or the forward, rewind processing of S450 isperformed.

With referring to FIG. 31, a case when the absolute value of the setplayback speed m is below “1”, namely, the normal playback, slowplayback, frame forward, frame rewind (S440) is explained. For the caseof forwarding or rewinding, which will be explained later, about 5frames are suitable to enable the user to sufficiently confirm visuallyas the number of frames g to be displayed per second. On the other hand,about 15 to 30 frames are suitable as the number of frames to bedisplayed per second for the normal playback in order to realize thesmooth playback. Generally, it is desirable to display all frameswithout omitting because the number of frames is set to 15 to 30 framesper second considering the smooth playback to be realized in the normalplayback speed for the digitallized moving picture data. It is alsodesirable that the frames are all displayed without being omitted forthe slow playback, frame forward, and frame rewind, naturally. Whenobtaining the images of the frames one by one by frame obtainment unit309 in the explanation given later at S450, the number of frames g thatcan be obtained per second is assumed to be 5 frames, but for the casein which the moving picture data, etc., compressed in MPEG is thetarget, when the continuous frames are continuously read and playedback, the number of frames that can be played back per second generallyincreases more than 5 frames. It is assumed that the playback at thespeed of 30 frames per second is possible when the continuous frames arecontinuously read and played back.

When the number of frames per second of the target moving picture datais assumed to be f, and the set playback speed to be m, then the timefor displaying 1 frame becomes 1/(f*m), therefore, frames can beobtained one after another at the time interval and displayed for thecase of normal speed playback or slow playback.

The processing of S440 is explained in concrete according to FIG. 31.

At S441, frame obtainment unit 309 reads frame F, first. Then, at S443,after the previous frame is displayed, the time 1/(f*m) is waited untilit is passed. At S445, moving picture display unit 307 displays frame F.After that, at S 449, the frame F to be displayed next is assumed to beF+1. For the case of frame forward or frame rewind, the processing waitsuntil the next button is pressed, and the next frame of the previousframe can be obtained each time when the button is pressed anddisplayed.

When the absolute value of the playback speed m set by playback speedset unit 301 exceeds “1”, that is, a case in which the forward or rewindis specified (S450) is explained next according to FIG. 32.

When the number of frames to be obtained per second by frame obtainmentunit 309 is assumed to be g, in other words, when the number of framesto be displayed per second during forwarding or rewinding is assumed tobe g, it means that frames can be switched at every 1/g second anddisplayed. Therefore, frame F is read (S451) by frame obtainment unit309. Then, after the previous frame is displayed, the time 1/g is waiteduntil it is passed (S453). Then, at S455, frame F is displayed by movingpicture display unit. When the number of currently displayed frame isassumed to be F0, the number of frame to be next displayed becomesF0+f*m/g (S457). However, in order to avoid the high priority framesfrom being omitted as explained previously, the processing is done asbelow.

From frame number F0+1 to frame number F0+1*m/g is set A as the searchscope, and a frame with high priority within the search scope isselected, and the selected frame is set as the frame to be nextdisplayed (S459). Because the frame that should be played back based onthe playback speed is F0+f*m/g, this frame is newly assumed to be F0.When there are a plurality of frames with high priority within thesearch scope, a frame that can be searched earlier than others isselected, and when the next selection scope is assumed to be from F+1 toF0+f*m/g, then another frame with high priority is to be selected in thenext search.

In this embodiment, the moving picture playback apparatus provided withframe obtainment unit 309 which obtains the image of the moving pictureframes, priority obtainment unit 305 which obtains the priority set forthe moving picture frames, playback speed set unit 301 which sets themoving picture playback speed, frame selection unit 303 whichdiscards/selects frames of the moving picture based on the playbackspeed set by playback speed set unit 301 and the priority of the movingpicture frames obtained by priority obtainment unit 305, and movingpicture display unit 307 which obtains the image of the frame selectedby frame selection unit 303 and obtains it from frame obtainment unit309 and displays is explained.

Embodiment 10

The moving picture playback apparatus which can switch the frame of themoving picture to high speed and display by means of the typical imageobtainment unit and the typical image storage unit is explained next.

FIG. 33 shows the configuration of the moving picture playback apparatusof Embodiment 10.

The moving picture playback apparatus of this embodiment is equippedwith a frame obtainment unit 309 for obtaining the image of the frame ofthe moving picture, a typical image obtainment unit 313 for obtainingthe image of the selected frame of the moving picture as a typicalimage, a typical image storage unit 311 for storing the typical imageobtained by typical image obtainment unit 313, a playback speed set unit301 for setting the moving picture playback speed, a frame selectionunit 303 for selecting the frame of the moving picture based on theplayback speed set by playback speed set unit 301, and a moving picturedisplay unit 315 for reading the image of the selected frame by frameselection unit 303 and displaying. A computer 320 is also illustrated.

This embodiment is especially effective when the moving picture datastored in a server computer is read from a client computer connected viathe network and playback. In that case, the previously mentionedcomputer 320 corresponds to the client computer, and typical imagestorage unit 311 corresponds to the memory of the client computer.

Although the previous Embodiment 9 is explained assuming that frameobtainment unit 309 can obtain the image of the specified frame at thespeed of 5 frames per second, the transfer speed may decline when themoving picture data is transferred via the network. If the transferspeed is assumed to be about 2 frames per second, the number of framesto be displayed per second decreases from 5 to 2 with the configurationof the moving picture playback apparatus of Embodiment 9, which has adefect of degrading efficiency in searching scenes.

This embodiment aims at overcoming this defect. In order not to decreasethe number of frames to be displayed per second in such a case, typicalimage storage unit 311 is provided for the main storage unit of a clientcomputer capable of reading and displaying the image at high speed forstoring the typical images.

The operation of each configuration element is explained below.

The operation for frame obtainment unit 309, playback speed set unit301, frame selection unit 303 is basically the, same as that of previousEmbodiment 9.

Typical image obtainment unit 313 obtains the image of the frameselected among the moving picture frames as the typical image. Aselected frame means about a frame the user selects from the movingpictures so as to be stored as the typical image. A case of selecting aframe corresponding to the priority set to the frame is assumed in thisembodiment. As a concrete method of obtaining a typical image, forinstance, there is a method of obtaining the data by obtaining the frameselected among the moving picture data stored in the server computer,etc., and then expanding it. In such a case, although obtaining thetypical image each time is time consuming and troublesome, only theframe number of the selected frame can be stored, therefore, the storagecapacity can be smaller compared to the case of storing the typicalimage itself. It is also possible to store the typical image datadefined separately from the moving picture data collectively as themoving picture data beforehand, and then read it out. As one example ofthe latter case, a format of storing the moving picture data and thetypical image data collectively in one file is shown in FIG. 34. In thisexample, the leading frame of each scene defined in the moving pictureis defined as the typical image. The leading frame of each scene is theframe with high priority indicated in the previous Embodiment 9. Thetypical image is defined for the major scene, medium scene, and smallscene, respectively. The meaning of the major scene, medium scene, andthe small scene is the same as the previous embodiment. The typicalimages n0, n1, n2, n3, and n4 of the major scene are positioned to thebeginning of the entire data, and the typical images n21, n22, n41, andn42 of the medium scene are positioned to the beginning of the data ofthe major scenes (scene 2, scene 4) to which each medium scene belongs,and the typical images of the small scene n211, n421 are positioned tothe beginning of the medium scenes (scene 2-1, scene 4-2) to which eachsmall scene belongs.

The case of using the format files shown in FIG. 34 is explained below.

Typical image obtainment unit 313 selects the frame of the necessarytypical image in accordance with the scene to be played back, obtainsthe selected typical image, and stores it in typical image storage unit311. For instance, when scene 2-1-0 is tried to be played back, n211which is the typical image of the small scene, n21 and n22, which arethe typical images of the medium scene, and n0; n1, n2, n3, and n4,which are the typical images of the major scene are obtained and storedin typical image storage unit 311. All of these typical image data arestored before the moving picture data of scene 2-1-0 in the format fileindicated in FIG. 34. Thus, typical image obtaining unit 313 can obtainthe typical image by reading the file in the forward direction and storeit in typical image storage unit 311. As a result, by the time when themoving picture data of the desired scene has been reached, the necessarytypical image for the scene is obtained in typical image storage unit311. When typical image storage unit 311 is incorporated in the internalmemory of the client computer, in this way, there is a merit of readingthe typical image faster than reading the typical image each time fromthe moving picture data stored in the external memory, for instance, thehard disk or the hard disk of the server computer, and therefore,efficient sense display and search is realized.

In a situation that the necessary typical image is obtained according toone of the scenes so far described, a case of trying to play back otherscenes is explained next.

Let us assume now that the scene to be played back is changed by theoperator, from scene 2-1-0 to scene 4-0. Typical images n211, n21, andn22 are no longer necessary for scene 4. Therefore, typical imageobtainment unit 313 deletes typical images n211, n21, and n22, andobtains typical images n41 and n42 corresponding to scene 4 newly andmakes typical image storage unit 311 add and store them.

In this way, for the scenes close to the currently displayed scene, eventhe frames with low priority are stored as the typical images, and forthe scenes far from the currently displayed scene, only the frames withhigh priority are stored as the typical images. By changing the typicalimages to be stored according to the position of the frame currentlydisplayed, the effective scene display and search are enabled byefficiently using typical image storage unit 311 which has a limitedstorage capacity.

The operation flows of the moving picture playback apparatus of thisembodiment are shown in FIG. 35 to FIG. 37. The explanation is givenaccording to the flow hereinbelow.

Because the signs to be used for FIG. 35 to FIG. 37 are the same asthose used for FIG. 30 to FIG. 32, their explanation is omitted.

Firstly, the explanation is given with referring to FIG. 35. At S471,“1” is set for the current frame position F and the original frameposition F0. Then, at S473, when the scene to which the current frameposition belongs has changed, the predetermined typical imagecorresponding to the scene is obtained by the typical image obtainmentunit and stored in the typical image storage unit. Then, at S475, theset playback speed m is obtained. Then, it is determined whether theabsolute value of the set playback speed exceeds “1” (S477), and basedon this, either the normal playback, frame forward, frame rewindprocessing of S480 or the forward or rewind processing of S490 isperformed.

Using FIG. 36, a case of when the absolute value of the set playbackspeed m is below “1”, namely, a case of the normal playback, slowplayback, frame forward, frame rewind, and so on (S480) is explained.

First, at S481, frame obtainment unit 309 reads the frame F. Then, atS483, after the previous frame is displayed, the time 1/(f*m) is waiteduntil it is passed. Then, at S485, the frame F is displayed by movingpicture display unit 315. Next, at S 487, the original frame position Fis assumed to be F0+1. Then, at S489, the frame F to be displayed nextis assumed to be F+1. For the frame forward or frame rewind, theprocessing waits until the button is pressed, and the next frame or theprevious frame can be obtained and displayed every time the button ispressed.

Then, a case in which the absolute value of the playback speed m set byplayback speed set unit 301 exceeds “1”, e.g., when the forwarding orrewinding is specified (S490) is explained according to FIG. 37.

When the number of frames obtained per second by frame obtainment unit309 is assumed to g, in other words, when the number of frames that canbe displayed per second during forwarding or rewinding is assumed to beg, then frames may be switched and displayed by every 1/g second.Therefore, typical images are read from the typical image storage unit(S491). Then, after the previous frame is displayed, the time 1/g iswaited until it is passed (S493). Then, at S495, the typical image readby the moving picture display unit is displayed as the frame F. Then,because after 1/g second, by which the next frame has to be displayed,the processing must be proceeded by f*m/g frame. Therefore, if thecurrently displayed frame number is F0, the frame number to be nextdisplayed becomes F0+f*m/g (S497). From the frame number F+1 throughframe number F0 are set as the search scope, and a frame with highpriority within the search scope is selected, and the selected frame isset as the frame F to be next displayed (S499).

Moving picture display unit 315 changes the reading destination of themoving picture data based on the set playback speed. When the setplayback speed exceeds the normal playback speed, e.g., when theforwarding or rewinding is specified, only the frame for which the highpriority is set may be displayed. Because the frames with high priorityare stored in typical image storage unit 311, data can be read fromtypical image storage unit 311 at high speed and displayed.

When the set playback speed is below the normal playback speed, e.g.,when the normal playback, frame forward, frame rewind, etc., isspecified, there is no need to switch frames at high speed, rather,frames have to be displayed accurately including the frames with lowpriority. Therefore, the image data is read by each frame from frameobtainment unit 309 and displayed.

Generally, when a specific frame is searched among the moving pictureframes. it is efficient that the positioning is first made close to thetarget frame by the high speed forwarding or rewinding, and then thepositioning operation is performed to the specific frame by the lowspeed frame forwarding or frame rewinding. Such operation is normallycarried out irrespective of the digital or analog moving picture data

In this embodiment, when the high speed forward or rewinding isperformed, the image is displayed at high speed by reading the datastored in typical image storage unit 311, and for the low speed frameforwarding or frame rewinding, the image is displayed accurately in theframe unit by reading the data directly from the moving picture storageunit by the frame obtainment unit. Thus, a remote client computer canaccess the moving picture data stored in the server computer, and asearch of the frame at an arbitrary speed forwarding or rewinding isrealized.

In this embodiment, the moving picture playback apparatus having frameobtainment unit 309 for obtaining the frame images in the movingpictures, priority obtainment unit 305 for obtaining the priority setfor the moving picture frames, typical image obtainment unit 313 forobtaining the image of the frame selected from the moving pictures asthe typical image, typical image storage unit 311 for storing thetypical image obtained by the previously mentioned typical imageobtainment unit 313, playback speed set unit 301 for setting the movingpicture playback speed, the frame selection unit fordiscarding/selecting the frame in the moving picture based on theplayback speed set by playback speed set unit 301 and the priority ofthe frame in the moving picture obtained by priority obtainment unit305, and moving picture display unit 315 for reading the image of theframe selected by frame selection unit 303 from typical image storageunit 311 and displaying it is explained.

Typical image storage unit 311 of the moving picture playback apparatusof this embodiment changes the frame to be stored as the typical imageaccording to the position of the frame currently displayed.

INDUSTRIAL APPLICABILITY

This invention has effects as hereinbelow described due to theconfiguration thus far set forth.

According to the first to third invention, because the segment of themoving picture is defined as a scene with the frame corresponding to theevents being shot as the end, the definition operation for the scene isnot required, thereby a scene (moving picture) to be displayed can beplayed back in a short time period.

According to the fourth invention, because the new scenes are playedback and edited by reading the scenes stored in the moving picturestorage unit based on the scene information stored in the sceneinformation storage unit, the moving pictures can be played back andedited quickly with an easy operation.

According to the fifth invention, because the typical frame of the scenedefined with the segment of the moving picture as the scene is read fromthe typical frame storage unit and displayed, there is no need ofstoring all moving pictures which are the target of editing by digitallycompressing in the moving picture storage unit, the contents of themoving pictures can be confirmed in a short time period.

According to the sixth invention, if the frame to be displayed is thetypical frame, the typical frame can be displayed, and if the frame tobe displayed is not the typical frame, then the closest typical frame isread from the previous frames from the frame storage unit and displayed,the contents of the scene can be confirmed in the short time period.

According to the seventh invention, if the frame to be displayed is notthe typical frame during stop, frame forward, or frame rewind, then theframe to be displayed is read from the moving picture storage unit anddisplayed thus the contents of the scene can be confirmed in the shorttime period.

According to the eighth invention, if the frame to be displayed is thetypical frame during stop, frame forward, or frame rewind, then theframe to be displayed is read from the typical frame storage unit anddisplayed, thus the contents of the scene can be confirmed in the shorttime period.

According to the eighth invention, if the frame to be displayed is thetypical frame during stop, frame forward, or frame rewind, then theframe to be displayed is read from the moving picture storage unit anddisplayed, thus the contents of the scene can be confirmed in the shorttime period.

According to the ninth invention, because the typical framedetermination unit selects a frame corresponding to the event detectedby the event detection unit as the typical frame for the scene includingthat frame, the scene definition and the typical frame reading can becompleted at the same time as the shooting, thus the moving pictures canbe played back instantaneously and the contents can be confirmed.

According to the tenth invention, new scenes can be played back andedited by reading the scenes stored in the moving picture storage unitbased on the typical frame stored in the typical frame storage unit, themoving pictures can be played back and edited quickly with an easyoperation.

According to the eleventh through thirteenth invention, because theinformation of the event detected by the event detection unit and theframe numbers obtained by the frame number obtainment unit are stored incorrelation, the index can be quickly displayed after the shooting ends,and therefore, the user can select the desired place of the movingpicture and confirm the contents quickly.

According to the fourteenth and fifteenth invention, because theexplanation sentence the user added to the event is stored in the eventinformation storage unit as the event information, the user can selectthe desired place of the moving picture with the explanation sentence asthe clue, as the explanation sentence is displayed as the index.

According to the sixteenth and seventeenth invention, because the eventdetection unit detects the approach, entry, or coming out of the presetshooting subject or shooting area, the places in the moving picturecorresponding to the preset target or area as the index item, the usercan select the desired place in the moving picture with this as theclue, and can confirm the contents quickly.

According to the eighteenth and nineteenth invention, because the imageshooting device control unit performs the control of the shooting devisecorresponding to the event detected by the event detection unit, thereis no need for the user to perform the control of the shooting devisemanually during shooting.

According to the twentieth invention, because the moving pictureplayback instruction unit instructs to playback the frame by indicatingthe frame number corresponding to the selected index item, the user canplayback the place in the moving picture corresponding to the item byselecting the arbitrary item of the index, the contents can be quicklyconfirmed.

According to the twenty-first invention, because the information of theevent detected by the event detection unit is expressed by the image orsound, and then stored by compositing with the moving picture shot bythe shooting device, the event information can be overposed on thescreen of the played back moving picture, and the user can find thedesired place in the moving picture with the displayed event informationas the clue, and the contents can be quickly confirmed.

According to the twenty-second invention, because the frame is selectedand displayed based on the priority set by the priority set unit and theplayback speed based on the playback speed set unit, forwarding orrewinding is enabled at an arbitrary speed without omitting the displayof the important frame in the moving picture, and the contents of themoving picture can be quickly confirmed.

According to the twenty-third through twenty-fifth invention, becausetypical image obtainment unit 313 obtains the typical image and storesin the typical image storage unit, and frame obtainment unit 309 readsthe typical image from the typical image storage unit and obtains theframes in the moving picture selected as the typical image can beswitched and displayed at high speed, the contents of the moving picturecan be quickly confirmed.

1. A moving picture playback apparatus, comprising: a priority set unitfor setting a priority for a frame of a moving picture composed of aplurality of continuous frames; a priority obtainment unit for obtainingthe priority set for the frame of the moving picture by the priority setunit; a playback speed set unit for setting a moving picture playbackspeed at which the moving picture is played back; a frame selection unitfor selecting a frame of the moving picture based on the playback speedset by the playback speed set unit and the priority set for the frame ofthe moving picture obtained by the priority obtainment unit; a frameobtainment unit for obtaining an image of the frame selected by theframe selection unit; and a moving picture display unit for displayingthe image of the frame obtained by the frame obtainment unit; andwherein the frame selected by the frame selection unit is registered ina frame list at a predetermined location so that frames, even with lowpriority that are close to the currently displayed frame and frames withonly high priority that are far from the currently displayed frame arestored as typical images.
 2. A moving picture playback apparatus,comprising: a priority set unit for setting a priority for a frame of amoving picture composed of a plurality of continuous frames; a frameobtainment unit for obtaining an image of the frame of the movingpicture; a priority obtainment unit for obtaining the priority set forthe frame of the moving picture by the priority set unit; a typicalimage obtainment unit for obtaining the image of the frame set a highpriority by the priority set unit as a typical image; a typical imagestorage unit for storing the typical image obtained by the typical imageobtainment unit; a playback speed set unit for setting a moving pictureplayback speed at which the moving picture is to be played back; a frameselection unit for selecting the frame of the moving picture based onthe playback speed set by the playback speed set unit and the priorityset for the moving picture frame obtained by the priority obtainmentunit; and a moving picture display unit for displaying the image of theframe selected by the frame selection unit from one of the frameobtainment unit and the typical image storage unit wherein the typicalimage storing unit changes the frame to be stored as the typical imageaccording to a timing relation between the frame of the typical imagestored by the typical image storage unit and the frame currentlydisplayed by the moving picture display unit so that frames, even withlow priority that are close to the currently displayed frame and frameswith only high priority that are far from the currently displayed frameare stored as typical images.
 3. The moving picture playback apparatusof claim 2, wherein the moving picture display unit reads and displaysthe image of the frame from the typical image storage unit when theplayback speed set by the playback speed set unit exceeds a normalplayback speed, and reads and displays the image of the frame from theframe obtainment unit when the playback speed is below the normalplayback speed.
 4. A moving picture playback apparatus, comprising: apriority set unit for setting a priority for a frame of a moving picturecomposed of a plurality of continuous frames; a frame obtainment unitfor obtaining an image of the frame of the moving picture; a priorityobtainment unit for obtaining the priority set for the frame of themoving picture by the priority set unit; a typical image obtainment unitfor obtaining the image of the frame set a high priority by the priorityset unit as a typical image; a typical image storage unit for storingthe typical image obtained by the typical image obtainment unit; aplayback speed set unit for setting a moving picture playback speed atwhich the moving picture is to be played back; a frame selection unitfor selecting the frame of the moving picture based on the playbackspeed set by the playback speed set unit and the priority set for themoving picture frame obtained by the priority obtainment unit; a movingpicture display unit for displaying the image of the frame selected bythe frame selection unit from one of the frame obtainment unit and thetypical image storage unit, and wherein the typical image storing unitchanges the frame to be stored as the typical image according to aposition of the frame currently displayed by the moving picture displayunit so that frames, even with low priority that are close to thecurrently displayed frame and frames with only high priority that arefar from the currently displayed frame are stored as typical images.